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DETAILED ACTION 

Response to Arguments 

1 . Applicant's arguments filed 08/31/10 have been fully considered but they are not 
persuasive. 

Applicant argues that neither D'hoore nor Riis nor Fabiani teach or suggest for 
each of the plurality of words, applying the phoneme map to each of the N first 
phoneme sequences for that word in order to translate the N first phoneme sequences 
in N second phoneme sequences, each of the N second phoneme sequences formed 
from phonemes associated with the mother tongue language (Amendment, pages 7 - 
10). 

The examiner disagrees, since Fabiani et al., disclose that "all the vocabulary 
items in the sub-set are further associated to transcriptions belonging to a common 
default language. Preferably, the sub-transcription units of the transcriptions from 
languages in the first sub-group of languages are mapped onto sub-transcription 
units of the default language... the sub-transcription unit(s) that does not have a 
speech model available in the collection of reference sub-transcription units is 
processed to derive a substitute sub-transcription unit from the collection of 
reference sub-transcription units that is acoustically similar to the sub- 
transcription unit(s) that does not have a speech model available. The sub- 
transcription unit(s) that does not have a speech model available is then replaced in 
the transcription by the substitute sub-transcription unit(s). The modified transcription 
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is then released. In the specific example where the sub-transcription units are 
phonemes, a nearest phoneme method is used for the substitution process. The 
nearest phoneme method is described in detail below in particular with reference to 
block 708 of FIG. 7. In its broad aspect, a nearest phoneme method maps phoneme A 
to a phoneme B that is acoustically similar to phoneme A residing in a collection of 
phonemes" (sub-transcriptions units in the first sub-groups of languages, which are 
phonemes that do not have specific models in the reference collection represent the 
plurality of words, wherein the collection of reference sub-transcriptions represent 
phonemes of the mother-tongue (default) language; paragraphs 33, 50, and 63; 
see also example of paragraph 71 ). 

Applicant argues that neither D'hoore nor Riis nor Fabiani teach or suggest for 
each word, two different phonetic transcripts are generated for each of the N different 
languages, including the N first phoneme sequences for the word, each formed from 
phoneme associated with one of the N different languages and the N second phoneme 
sequences for the word, each formed by applying the phoneme map to translate one of 
the N first phoneme sequences formed from phonemes associated with one of the N 
different languages into a sequence of phonemes associated with the mother tongue 
language (Amendment, pages 7-10). 

The examiner disagrees, since Fabiani et al., disclose that "all the vocabulary 
items in the sub-set are further associated to transcriptions belonging to a common 
default language. Preferably, the sub-transcription units of the transcriptions from 
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languages in the first sub-group of languages (N first phoneme sequences for the 
word) are mapped onto sub-transcription units of the default language (N second 
phoneme sequences for the word)... the sub-transcription unit(s) that does not have a 
speech model available in the collection of reference sub-transcription units is 
processed to derive a substitute sub-transcription unit from the collection of 
reference sub-transcription units that is acoustically similar to the sub- 
transcription unit(s) that does not have a speech model available. The sub- 
transcription unit(s) that does not have a speech model available is then replaced in 
the transcription by the substitute sub-transcription unit(s). The modified transcription 
is then released. In the specific example where the sub-transcription units are 
phonemes, a nearest phoneme method is used for the substitution process. The 
nearest phoneme method is described in detail below in particular with reference to 
block 708 of FIG. 7. In its broad aspect, a nearest phoneme method maps phoneme A 
to a phoneme B that is acoustically similar to phoneme A residing in a collection of 
phonemes" (sub-transcriptions units in the first sub-groups of languages represent the 
N first phoneme sequences for the word, and ; paragraphs 33, 50, and 63; see also 
example of paragraph 71 ). 

Applicant argues that claims 22 - 29 are statutory since recite now an automatic 
language recognizing apparatus, including computer program modules encoded in 
tangible computer readable media (Amendment, page 6). 



Application/Control Number: 10/501,700 Page 5 

Art Unit: 2626 

The examiner disagrees, since computer readable medium is not defined in the 
specification as non-transitory medium. Thus, the 101 rejection, regarding claims 22 - 
29 is maintained. 



Claim Rejections - 35 USC § 101 

2. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

Claims 22 - 29 are rejected under 35 U.S.C. 1 01 because the claimed invention 
is directed to non-statutory subject matter. 

Although Claims 22 - 29 appear to fall within a statutory category (i.e., 
apparatus), Claims 22 - 29 encompass nothing more than logic/software modules as 
per the specification ("The determination of the phonetic transcripts in the first step of 
the method takes place preferably by means of at least one neural network. Neural 
networks have proved suitable for determining phonetic transcripts from written words, 
because they produce good results with regards to accuracy, and particularly with 
regard to the speed of processing and can be easily implemented, particularly in 10 
software", Page 9, lines 5 - 10). Thus, Claims 22 - 29 are directed to non-statutory 
subject matter because their scope includes a computer program embodiment, an 
abstract data structure which does not fall within one of the four statutory categories 
(i.e., it is directed to a program perse). See also MPEP § 2106.IV.B.1 .a. Data structures 
not claimed as embodied in computer readable media are descriptive material perse 
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and are not statutory because they are not capable of causing functional change in the 
computer. See, e.g., Warmerdam, 33 F.3d at 1361, 31 USPQ2d at 1760 (claim to a data 
structure perse held nonstatutory). Such claimed data structures do not define any 
structural and functional interrelationships between the data structure and other claimed 
aspects of the invention, which permit the data structure's functionality to be realized. In 
contrast, a claimed computer readable medium encoded with a data structure defines 
structural and functional interrelationships between the data structure and the computer 
software and hardware components which permit the data structure's functionality to be 
realized, and is thus statutory. Similarly, computer programs claimed as computer 
listings per se, i.e., the descriptions or expressions of the programs are not physical 
"things." They are neither computer components nor statutory processes, as they are 
not "acts" being performed. Such claimed computer programs do not define any 
structural and functional interrelationships between the computer program and other 
claimed elements of a computer, which permit the computer program's functionality to 
be realized. 

Claims 22 - 29 are directed to a computer readable medium storing processor 
executable instructions that is not limited to a non-transitory, and thus, statutory 
medium. The scope of "computer-readable medium" is not defined in the specification 
may include signal-based mediums such as "signals used to propagate instructions", 
"carrier waves/pulses", since the specification discloses "The phoneme sequence 
determination and the succeeding mapping or depiction normally run offline on a device, 
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for example a mobile telephone, a personal digital assistant or personal computer with 
corresponding software, and are therefore time uncritical. The resources required for 
this can be held in an internal/external memory"; (see Specification, Page 8, lines 1-7). 
A signal does not fall within one of the four statutory categories of invention (i.e., 
process, machine, manufacture, or composition of matter) because it is an ephemeral, 
transient signal and thus is non-statutory. Since the scope of "computer-readable 
medium" includes these non-statutory instances, claims 22 - 29 are directed to non- 
statutory subject matter. 



Claim Rejections - 35 USC § 103 

3. The text of those sections of Title 35, U.S. Code not included in this action can 
be found in a prior Office action. 

4. Claims 12, 13, 20, 21, 22, 28, and 29 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over D'hoore et al., (US Patent 6,085,160) in view of Riis et al., (US 
PAP 2003/0050779); and further in view of Fabiani et al., (US PAP 2002/0173945). 

As per claims 12, and 22, D'hoore et al., teach a method/apparatus for 
automated language recognition of words from different languages said method 
embodied as computer program instructions encoded in tangible computer readable 
media comprising the steps of: 

(a) loading a phoneme set associated with a language specified as a mother 
tongue into a mother tongue language recognizer ("subword units in a first language"; 
col.2, lines 7- 14); 
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(b) for each of a plurality of words, determining phonetic transcripts for the word 
for N various languages not specified as the mother tongue to generate N first phoneme 
sequences for the word corresponding to N first pronunciation variants, each of the N 
first phoneme sequences formed from phonemes associated with one of the N different 
languages ("generate several phonetic transcriptions"; col. 8, lines 13-15); 

However, D'hoore et al., do not specifically teach determining a phoneme map by 
mapping the generated first phoneme sequences of each of said N languages to a 
relevant phoneme set of the mother tongue; for each of the plurality of words, applying 
the phoneme map to each of the N first phoneme sequences for that word in order to 
translate the N first phoneme sequences in N second phoneme sequences, each of the 
N second phoneme sequences formed from phonemes associated with the mother 
tongue language; such that for each word, two different phonetic transcripts are 
generated for each of the N different languages, including the N first phoneme 
sequences for the word, each formed from phoneme associated with one of the N 
different languages and the N second phoneme sequences for the word, each formed 
by applying the phoneme map to translate one of the N first phoneme sequences 
formed from phonemes associated with one of the N different languages into a 
sequence of phonemes associated with the mother tongue language; processing said N 
second phoneme sequences with the phoneme set associated with the language 
specified as the mother tongue to identify at least one of a matching word and a similar 
word. 
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Fabiani et al., disclose that the all the vocabulary items in the sub-set are further 
associated to transcriptions belonging to a common default language. Preferably, the 
sub-transcription units of the transcriptions from languages in the first sub- 
group of languages are mapped onto sub-transcription units of the default 
language... the sub-transcription unit(s) that does not have a speech model available 
in the collection of reference sub-transcription units is processed to derive a 
substitute sub-transcription unit from the collection of reference sub- 
transcription units that is acoustically similar to the sub-transcription unit(s) that 
does not have a speech model available. The sub-transcription unit(s) that does not 
have a speech model available is then replaced in the transcription by the substitute 
sub-transcription unit(s). The modified transcription is then released. In the specific 
example where the sub-transcription units are phonemes, a nearest phoneme method 
is used for the substitution process. The nearest phoneme method is described in 
detail below in particular with reference to block 708 of FIG. 7. In its broad aspect, a 
nearest phoneme method maps phoneme A to a phoneme B that is acoustically similar 
to phoneme A residing in a collection of phonemes (sub-transcriptions units in the first 
sub-groups of languages, which are phonemes that do not have specific models in the 
reference collection represent the plurality of words, wherein the collection of 
reference sub-transcriptions represent phonemes of the mother-tongue (default) 
language; paragraphs 33, 50, and 63; see also example of paragraph 71). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to use phonemes mappings as taught by Fabiani et al., in 
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D'hoore et al., because that would help generate suitable pronunciations of vocabulary 
items (Fabiani et al., paragraph 1). 

However, D'hoore et al., in view of Fabiani et al., do not specifically teach 
processing said N second phoneme sequences with the phoneme set associated with 
the language specified as the mother tongue to identify at least one of a matching word 
and a similar word. 

Riis et al., teach capturing both inter- and intra-language pronunciation variations 
which is ideal for multilingual speaker independent speech recognition systems; 
generating pronunciations in response to said sequences of multilingual phoneme 
symbols, and comparing said pronunciations with the acoustic input in order to find a 
match (Abstract, lines 7-10; paragraph 1 5, lines 8-11). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to generate inter- and intra-language pronunciation 
variations as taught by Riis et al., in D'hoore et al., in view of Fabiani et al., because that 
would help improve recognition performance (paragraph 67). 

As per claim 1 3, D'hoore in view of Riis et al., further in view of Fabiani et al., 
further disclose adding the N second phoneme sequences for each word in a language 
recognition vocabulary located in the mother tongue language recognizer (Riis et 
al., "generating a sequences of multilingual phoneme symbols"; paragraph 15). 
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As per claims 20, 21 , 28, and 29, D'hoore in view of Riis et al., further in view of 
Fabiani et al., further disclose determining the phonetic transcripts of each word for N 
various languages not specified as the mother tongue is performed by at least one 
neural network; processing said N second phoneme sequences with the phoneme set 
associated with the language specified as a mother tongue is performed using a Hidden 
Markov Model (Riis et al.,"The acoustic phoneme models where based on a HMM/NN 
hybrid known as Hidden Neural Networks (HNN)'; paragraph 53, lines 1 - 3). 

5. Claims 14-16, and 23 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over D'hoore et al., (US Patent 6,085,160) in view of Riis et al., (US PAP 
2003/0050779), further in view of Fabiani et al., (US PAP 2002/0173945), and further in 
view of Bub et al., (US Patent 6,460,017). 

As per claims 14, and 23, D'hoore et al., in view of Riis et al., further in Fabiani et 
al., do not specifically teach determining distances to the N second pronunciation 
variants based at least on the processed N second phoneme sequences 

Bub et al., teach measuring the distance or determining the similarity of two 
phonemes models of the same sound from different languages (col.1 1 , lines 45 - 48). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to measure the distance between phonemes as taught by 
Bub et al., in D'hoore et al., in view of Riis et al., further in Fabiani et al., because that 
would help improve recognition performance (Riis et al., paragraph 67). 
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As per claim 15, Riis et al., further disclose classifying each N second phoneme 
sequences ("inter- and intra-language pronunciation variations"; Abstract, lines 7-10). 

However, D'hoore et al., in view of Riis et al., further in view of Fabiani et al., do 
not specifically teach identifying respective distances. 

Bub et al., teach measuring the distance or determining the similarity of two 
phonemes models of the same sound from different languages (col.1 1 , lines 45 - 48). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to measure the distance between phonemes as taught by 
Bub et al., in D'hoore et al., in view of Riis et al., further in view of Fabiani et al., 
because that would help improve recognition performance (Riis et al., paragraph 67). 

As per claim 16, Bub et al., further disclose eliminating any N second phoneme 
sequences that do not meet or exceed a predetermined threshold ("distance threshold"; 
col.1 2, lines 51, and 52). 

6. Claims 17, 24, and 25 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over D'hoore et al., (US Patent 6,085,1 60) in view of Riis et al., (US PAP 
2003/0050779), further in view of Fabiani et al., (US PAP 2002/0173945), further in view 
of Bub et al., (US Patent 6,460,017), and further in view of Brill et al., (US Patent 
7,047,493). 
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As per claims 17, 24, D'hoore et al., in view of Riis et al., further in view of 
Fabiani et al., and further in view of Bub et al., do not specifically teach that the 
distances are Levenshtein distances. 

Brill et al., teach using Levenshtein distance (col. 3, line 31). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to use Levenshtein distance as taught by Brill et al., in 
D'hoore et al., in view of Riis et al., further in view of Fabiani et al., and further in view of 
Bub et al., because that would help improve recognition performance (Riis et al., 
paragraph 67). 

As per claim 25, Bub et al., further disclose eliminating any N second phoneme 
sequences that do not meet or exceed a predetermined threshold ("distance threshold"; 
col. 12, lines 51, and 52). 

7. Claims 18, 19, 26, and 27 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over D'hoore et al., (US Patent 6,085,160) in view of Riis et al., (US PAP 
2003/0050779), further in view of Fabiani et al., (US PAP 2002/0173945), and further in 
view of Harengel et al., (US PAP 2004/0039570). 

As per claims 18, 19, 26, and 27, D'hoore et al., in view of Riis et al., further in 
view of Fabiani et al., do not specifically teach determining the probabilities that each 
word for N various languages not specified as the mother tongue belong to a specified 
set of languages, said step of determining probabilities occurring before step (a); and 
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eliminating languages from said specified set that do not exceed a predetermined 
threshold. 

Harengel et al., teach if the probability coefficient for the assignment of a word to 
at least one language exceeds the threshold value, the grapheme-phoneme assignment 
which corresponds to the respective word is supplemented in the pronunciation lexicon 
(paragraph 10, lines 8-13). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to determine a probability coefficient of a word as taught 
by Harengel et al., in D'hoore et al., in view of Riis et al., further in view of Fabiani et al., 
because that would help improve recognition performance (Riis et al., paragraph 67). 

Conclusion 

8. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See M PEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
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the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

9. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to LEONARD SAINT CYR whose telephone number is 
(571) 272-4247. The examiner can normally be reached on Mon- Friday. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Richemond Dorvil can be reached on (571) 272-7602. The fax phone 
number for the organization where this application or proceeding is assigned is (571)- 
273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or (571)-272-1000. 
LS 

11/07/10 

/Leonard Saint-Cyr/ 
Examiner, Art Unit 2626 



